Trust Region Newton Method for Large-Scale Logistic Regression
نویسندگان
چکیده
Large-scale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the log-likelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also extend the proposed method to large-scale L2-loss linear support vector machines (SVM).
منابع مشابه
A Trust Region Algorithm for Solving Nonlinear Equations (RESEARCH NOTE)
This paper presents a practical and efficient method to solve large-scale nonlinear equations. The global convergence of this new trust region algorithm is verified. The algorithm is then used to solve the nonlinear equations arising in an Expanded Lagrangian Function (ELF). Numerical results for the implementation of some large-scale problems indicate that the algorithm is efficient for these ...
متن کاملA limited memory adaptive trust-region approach for large-scale unconstrained optimization
This study concerns with a trust-region-based method for solving unconstrained optimization problems. The approach takes the advantages of the compact limited memory BFGS updating formula together with an appropriate adaptive radius strategy. In our approach, the adaptive technique leads us to decrease the number of subproblems solving, while utilizing the structure of limited memory quasi-Newt...
متن کاملA Fast Trust-Region Newton Method for Softmax Logistic Regression
With the emergence of big data, there has been a growing interest in optimization routines that lead to faster convergence of Logistic Regression (LR). Among many optimization methods such as Gradient Descent, Quasi-Newton, Conjugate Gradient, etc., the Trust-region based truncated Newton method (TRON) algorithm has been shown to converge the fastest. The TRON algorithm also forms an important ...
متن کاملA Two-dimensional Trust-region Method for Large Scale Bound-constrained Nonlinear Systems
We design a well-developed algorithm for solving large bound-constrained systems. The method combines a subspace trust-region strategy with a Newton-Krylov method and with a strategy for handling the bounds. To solve the trust-region problem efficiently, we look for an approximate solution and do not require the exact solution of the Newton equation. In addition, a strategy for obtaining an imp...
متن کاملInexact Newton Methods Applied to Under – Determined Systems by Joseph P . Simonis A Dissertation
Consider an under-determined system of nonlinear equations F (x) = 0, F : IR → IR, where F is continuously differentiable and m > n. This system appears in a variety of applications, including parameter–dependent systems, dynamical systems with periodic solutions, and nonlinear eigenvalue problems. Robust, efficient numerical methods are often required for the solution of this system. Newton’s ...
متن کامل